Get the Materials

git clone https://github.com/ypeleg/Deep-Learning-Keras-Tensorflow-PyCon-Israel-2017

Tentative Outline

Outline at a glance

Part I: Introduction
- Intro to Deep Learning and ANN
  - Perceptron and MLP
- naive pure-Python implementation
  - fast forward, sgd, backprop
- Intro to Tensorflow
  - Model + SGD with Tensorflow
- Introduction to Keras
  - Overview and main features
    - Keras Backend
    - Overview of the core layers
  - Multi-Layer Perceptron and Fully Connected
    - Examples with keras.models.Sequential and Dense
    - HandsOn: FC with keras
Part II: Supervised Learning and Convolutional Neural Nets
- Intro: Focus on Image Classification
- Intro to ConvNets
  - meaning of convolutional filters
    - examples from ImageNet
  - Visualising ConvNets
- Advanced CNN
  - Dropout
  - MaxPooling
  - Batch Normalisation
- HandsOn: MNIST Dataset
  - FC and MNIST
  - CNN and MNIST
- Deep Convolutiona Neural Networks with Keras (ref: keras.applications)
  - VGG16
  - VGG19
  - ResNet50
- Transfer Learning and FineTuning
- Hyperparameters Optimisation
Part III: Unsupervised Learning
- AutoEncoders and Embeddings
- AutoEncoders and MNIST
  - word2vec and doc2vec (gensim) with keras.datasets
  - word2vec and CNN
Part IV: Recurrent Neural Networks
- Recurrent Neural Network in Keras
  - SimpleRNN, LSTM, GRU
PartV: Additional Materials:
- Quick tutorial on theano
- Perceptron and Adaline (pure-python) implementations
- MLP and MNIST (pure-python)
- LSTM for Sentence Generation
- Custom Layers in Keras
- Multi modal Network Topologies with Keras

Requirements

This tutorial requires the following packages:

Python version 3.5
- Python 3.4 should be fine as well
- likely Python 2.7 would be also fine, but who knows? :P
numpy version 1.10 or later: http://www.numpy.org/
scipy version 0.16 or later: http://www.scipy.org/
matplotlib version 1.4 or later: http://matplotlib.org/
pandas version 0.16 or later: http://pandas.pydata.org
scikit-learn version 0.15 or later: http://scikit-learn.org
keras version 2.0 or later: http://keras.io
tensorflow version 1.0 or later: https://www.tensorflow.org
ipython/jupyter version 4.0 or later, with notebook support

(Optional but recommended):

pyyaml
hdf5 and h5py (required if you use model saving/loading functions in keras)
NVIDIA cuDNN if you have NVIDIA GPUs on your machines. https://developer.nvidia.com/rdp/cudnn-download

The easiest way to get (most) these is to use an all-in-one installer such as Anaconda from Continuum. These are available for multiple architectures.

Python Version

I'm currently running this tutorial with Python 3 on Anaconda



In [1]:

    
!python --version









    



Python 3.5.2

Configure Keras with tensorflow

1) Create the keras.json (if it does not exist):

touch $HOME/.keras/keras.json

2) Copy the following content into the file:

{
    "epsilon": 1e-07,
    "backend": "tensorflow",
    "floatx": "float32",
    "image_data_format": "channels_last"
}



In [2]:

    
!cat ~/.keras/keras.json









    



{
	"epsilon": 1e-07,
	"backend": "tensorflow",
	"floatx": "float32",
	"image_data_format": "channels_last"
}

Test if everything is up&running

1. Check import



In [3]:

    
import numpy as np
import scipy as sp
import pandas as pd
import matplotlib.pyplot as plt
import sklearn



In [4]:

    
import keras









    



Using TensorFlow backend.

2. Check installeded Versions



In [5]:

    
import numpy
print('numpy:', numpy.__version__)

import scipy
print('scipy:', scipy.__version__)

import matplotlib
print('matplotlib:', matplotlib.__version__)

import IPython
print('iPython:', IPython.__version__)

import sklearn
print('scikit-learn:', sklearn.__version__)









    



numpy: 1.11.1
scipy: 0.18.0
matplotlib: 1.5.2
iPython: 5.1.0
scikit-learn: 0.18



In [6]:

    
import keras
print('keras: ', keras.__version__)

# optional
import theano
print('Theano: ', theano.__version__)

import tensorflow as tf
print('Tensorflow: ', tf.__version__)









    



keras:  2.0.2
Theano:  0.9.0
Tensorflow:  1.0.1

If everything worked till down here, you're ready to start!

Deep Learning Tutorial with Keras and Tensorflow